Data Analysis Processes and Techniques for Validation of Wearable Technology: An Example

Topics in Exercise Science and Kinesiology Volume 3: Issue 1, Article 10, 2022. With wearable technology growing in popularity and sophistication, there remains a need to determine the validity of these devices by independent observers. Validation studies of wearable technology can involve large amounts of data, with data preparation techniques that are not always clearly established. This can make attempts to reproduce the results difficult and does not allow researchers to gain guidance in how to perform their own analyses if they wanted to perform a similar study. Therefore, this paper details the process that was utilized to prepare and analyze the accuracy of several heart rate monitors during mountain biking and can be used as a possible guide to researchers looking to perform similar analyses. We also detail the software used and discuss possible alternatives

Assessing the Validity and Reliability of Several Heart Rate Monitors in Wearable Technology While Mountain Biking

This study sought to assess the validity and reliability of several heart rate (HR) monitors during mountain biking (MTB), compared to the Polar H7® HR monitor, used as the criterion device. Methods: A total of 20 participants completed two MTB trials wearing 6 HR monitors (1 criterion, 5 test devices). HR was recorded on a second-by-second basis for all devices analyzed. After data processing, validity measures were calculated, including 1. Error analysis: mean absolute percentage errors (MAPE), mean absolute error (MAE), and mean error (ME), and 2. Correlation analysis: Lin’s concordance correlation coefficient (CCC) and Pearson’s correlation coefficient ®. Validity was determined for overall HR as well as stratified HR data based on 5 HR zones. Thresholds for validity were set at MAPE0.7. Reliability measures were also determined comparing trial 1 to trial 2 via two statistical tests: 1. Intra-class correlation coefficient (ICC) and 2. Coefficient of variation (CV). The predetermined reliability threshold was set at an ICC of \u3e0.7 and a C

Establishing a Methodology for Conducting a Rapid Review on Wearable Technology Reliability and Validity in Applied Settings

Wearable technology is becoming increasingly popular with new devices entering the market frequently, but with disproportionality less research being conducted on the reliability and validity in field settings, outside of the laboratory. This paper details the methodology of gathering research based on the principals of a rapid review, using these points of application: inclusion criteria of studies (databases), search terms (data to be extracted), and how research will be conducted (narrowing articles and assessing bias). Point of application #1: Inclusion criteria are described well as potential databases used, allowing a means of not only systematically gathering information, but also the ability to check the status of the literature regarding wearable technology. Point of application #2: Specific search terms are outlined, and the data fields to be extracted are described. Point of application #3: Mechanism of the rapid review search explained, how articles are to be included and excluded, and how bias will be assessed

The Risk of Bias in Validity and Reliability Studies Testing Physiological Variables using Consumer-Grade Wearable Technology: A Systematic Review and WEAR-BOT Analysis

INTRODUCTION: Wearable technology is a quickly evolving field, and new devices with new features to measure/estimate physiological variables are being released constantly. Despite their use, the validity of the devices are largely unknown to the users or researchers, and the quality of the studies that do test validity and reliability vary widely. PURPOSE: Therefore, the purpose of this systematic review was to review the current validity and reliability literature concerning consumer-grade wearable technology measurements/estimates of physiological variables during exercise. Additionally, we sought to perform risk of bias assessments utilizing the novel WEArable technology Risk of Bias and Objectivity Tool (WEAR-BOT). METHODS: This review was conducted following PRISMA guidelines, searching 3 databases: Google Scholar, Scopus, and SPORTDiscus. After screening, 46 papers were identified that met the pre-determined criteria. Then data was extracted and risk of bias assessment performed by independent researchers. Descriptive statistics, weighted averages of mean absolute percentage error (MAPE) and Pearson correlations were calculated. Sample size statistics were performed utilizing the lower 95% confidence interval of the weighted correlation average. RESULTS: Of the 46 papers reviewed, 44 performed validity testing, while 9 performed reliability. The weighted average for MAPE was 12.48% for heart rate (HR) and 30.70% for energy expenditure (EE). The weighted average for Pearson correlations was 0.737 for HR and 0.672 for EE. Risk of bias assessment of validity studies resulted in 30/44 studies being classified as having a “High Risk of Bias”, and 14/44 having “Some Risk of Bias”. None had a “Low Risk of Bias”, according to the novel WEAR-BOT. For reliability studies, 7/9 were classified as “High Risk of Bias”, 2 as “Some Risk of Bias”, and 0 as “Low Risk of Bias”. CONCLUSION: The risk of bias assessment and descriptive statistics paint a troubling picture of the overall state of validity and reliability studies. Statistical analyses, methods, and reporting vary excessively. This review and associated WEAR-BOT analysis can be used by researchers to help standardize methodology, analytics, and reporting of validation and reliability studies of consumer-grade wearable technology

A Systematic Review of the Effects of Meditative and Mindful Walking on Mental and Cardiovascular Health

International Journal of Exercise Science 15(2): 1692-1734, 2022. Meditative and mindful exercise are types of physical exercise during which people pay attention, on purpose, to each new present moment without judging their experience. The goal is to apply an accepting awareness of the environment, bodily sensations, thoughts, and emotions without labeling them (e.g., good or bad). The literature centers on qigong, tai chi, and yoga, which are types of mindful exercise that improve mental and cardiovascular health. It is unclear if meditative and mindful walking also improve these health domains. To the authors’ knowledge, this question has not been addressed by a published systematic review. The purpose of this systematic review without a meta-analysis was to synthesize the literature on meditative and mindful walking to determine their effects on mental and cardiovascular health. The protocol follows the PRISMA guidelines, is registered in PROSPERO (CRD42021241180), and is published elsewhere in a peer-reviewed journal. The systematic review contains 14 studies that had various populations, interventions, and outcomes. In 13 studies, the interventions statistically significantly improved scores on at least one outcome of mental or cardiovascular health (e.g., affect, anxiety, depression, distress, state mindfulness, stress, blood pressure, and six-minute walk distance). The improved outcomes should be interpreted cautiously because their clinical meaningfulness is unclear, and the studies had severe methodological limitations. Determining if meditative and mindful walking meaningfully improve mental and cardiovascular health will require randomized controlled trials that use rigorous designs, transparent protocols, and clinically meaningful outcomes that indicate physical function, mental wellbeing, morbidity, and mortality

Comparing Exercise Intensity as a Percentage of the Age-Estimated Heart Rate Max Among Walking, Jogging, and Skipping

BACKGROUND: Heart rate (HR) intensity in walking and running has been extensively studied. However, exploring the intensities of other activities such as skipping has been skipped over. Skipping is a playful activity usually performed in short bouts. The intensity and feasibility of skipping for several minutes is unclear. Studying HR during skipping may reveal that it is a novel and useful form of aerobic exercise. PURPOSE: The aim of this study was to compare HR intensity among walking, running, and skipping. METHODS: Ten participants gave verbal and written consent and self-reported biological sex, age, height, and mass (5 male, 5 female; 26.90 ± 9.43 yrs; 168.66 ± 9.37 cm; 72.64 ± 7.73 kg). Participants then wore a Polar H10 HR monitor with chest strap to record mean HR and max HR during the protocol: 5-min self-paced walk, 5-min seated rest, 5-min self-paced run, 5-min seated rest, and 5-min self-paced skip. Mean HR and mean HR as a percent of age-predicted max (%max) were compared across the three activities by using two separate one-way repeated-measures ANOVAs. Population effect sizes were estimated as partial omega squared (ωp2; large effect \u3e 0.14). For both ANOVAs, the post-hoc tests were pairwise comparisons among the three activities by using dependent-samples t-tests with Bonferroni adjustments. The α-level for all statistical analyses was 0.05. RESULTS: Both mean HR and %max significantly differed among the three activities (mean HR: F = 145.62, p \u3c 0.001, ωp2 = 0.91; %max: F = 162.57, p \u3c 0.001, ωp2 = 0.92). Mean HR was 103 ± 17 bpm during walking (%max = 54.2 ± 8.2%), 155 ± 17 bpm during running (%max = 81.6 ± 6.7%), and 170 ± 20 bpm during skipping (%max = 89.6 ± 8.5%). Mean HR and %max were significantly higher during skipping than walking (+67 bpm/+35%, p \u3c 0.001,) and running (+15 bpm/+8%, p \u3c 0.001). CONCLUSION: The data suggest that skipping is a significantly more intense exercise than walking and running when performed for several minutes. Thus, skipping can be used as a form of vigorous exercise. Participants’ high HR during skipping may have been caused by the novelty of the exercise. Skipping intensity in avid skippers should be a topic of focus in our future research

Efficient Method of Delivery for Powdered Supplement or Placebo for an Outdoor Exercise Investigation

Researchers often encounter issues while attempting to deliver complete doses of a desired supplement/placebo when conducting research in outdoor environments. The problem faced by our research team was how to efficiently deliver 6.4 grams of Beta Alanine (BA) powder to subjects while conducting a hiking study on a windy trail in Southern Utah. To minimize the potential impact of the weather, and to maximize the efficiency of delivery, we determined a premixed bolus of BA contained in individual commercially available 8oz water bottles was the most efficient delivery method. The purpose of this article is to detail the development and implementation of this method. Point of Application 1: Testing solubility, and efficacy of dissolved BA Point of Application 2: Producing a large volume of pre-mixed supplement-bottles and placebo-bottles Point of Application 3: On site administration of supplement and placeb

Validity and Reliability of Physiological Data in Applied Settings Measured by Wearable Technology: A Rapid Systematic Review

The purpose of this review was to evaluate the current state of the literature and to identify the types of study designs, wearable devices, statistical tests, and exercise modes used in validation and reliability studies conducted in applied settings/outdoor environments. This was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. We identified nine articles that fit our inclusion criteria, eight of which tested for validity and one tested for reliability. The studies tested 28 different devices with exercise modalities of running, walking, cycling, and hiking. While there were no universally common analytical techniques used to measure accuracy or validity, correlative measures were used in 88% of studies, mean absolute percentage error (MAPE) in 75%, and Bland–Altman plots in 63%. Intra-class correlation was used to determine reliability. There were not any universally common thresholds to determine validity, however, of the studies that used MAPE and correlation, there were only five devices that had a MAPE of... (see full abstract in article)

The Validity and Reliability of the Garmin Instinct in Measuring Heart Rate, Energy Expenditure, and Steps During Skipping

As the use of wearable technology to monitor physical activity increases, assessment of the validity and reliability of these devices are needed. A popular device brand is Garmin. Although not a common physical activity, skipping can be included in dynamic warm-ups. PURPOSE: Therefore, the purpose of this study was to determine the validity and reliability of the Garmin Instinct in measuring caloric energy expenditure (EE), average heart rate (HR), and steps while skipping. METHODS: Ten participants (5 female, age: 27±9 years) skipped at a self-selected pace for five minutes. During that time, HR, EE, and steps were measured by the Polar H10, Cosmed K5, and manual counting, respectively. Two Garmin Instincts simultaneously tracked all three variables. A step was defined as any time the foot leaves and hits the ground (stride x 4). Data was input into Google Sheets and summary statistics, t-test with Bonferonni corrections, and mean absolute percentage error (MAPE) were calculated. Additional validity and reliability tests were run in jamovi, including Lin’s concordance correlation coefficient (CCC), TOST tests, Bland-Altman bias, coefficient of variation (CV), and intraclass correlation coefficient (ICC). The pre-established validity criteria are as follows: CCC \u3e 0.7 and MAPE \u3c 10%. The pre-established reliability criteria are as follows: CV \u3c 10% and ICC \u3e 0.7. RESULTS: The Garmin Instinct had a MAPE of 19.2%, 28.5%, and 53.2% for HR, EE, and steps, respectively. It had a CCC of .06, .21, and .01 for HR, EE, and steps, respectively. The 2-tailed paired t-tests with corrections for multiple comparisons was significant for HR and steps. The TOST tests were violated for all 3 measurements (HR, EE, and steps). Bland-Altman analysis produced a bias estimate of 34.0, 0.6, and 1100 for HR, EE, and steps, respectively. The Garmin produced a CV of 11.2%, 14.8%, and 6.6% for HR, EE, and steps, respectively. It produced an ICC of .51, .64, and .81 for HR, EE, and steps, respectively. CONCLUSION: The Garmin Instinct did not meet the pre-established validity criteria for any measure (HR, EE, or steps). However, it did meet the pre-established reliability criteria for steps but not for HR or EE. Therefore, the Garmin Instinct cannot be expected to produce accurate estimates of HR, EE, or steps during skipping

A Protocol and Novel Tool for Systematically Reviewing the Effects of Mindful Walking on Mental and Cardiovascular Health

To our knowledge, no published systematic review has described the effects of mindful walking on mental and cardiovascular health. We have aimed to fill this gap by first establishing our systematic review protocol. Our protocol was adapted from the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and is registered in PROSPERO (Registration Number: CRD42021241180). The protocol is described step-by-step in this paper, which we wrote to achieve three objectives: to adhere to the best practices stated in the PRISMA guidelines, to ensure procedural transparency, and to enable readers to co-opt our protocol for future systematic reviews on mindful walking and related topics. To achieve our third objective, we provide and explain a novel tool we created to track the sources we will find and screen for our review. Ultimately, the protocol and novel tool will lead to the first published systematic review about mindful walking and will also facilitate future systematic reviews